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Abstract 


The  Software  Acquisition  Improvement  Framework  (SAIF)  is  a  computer-aided  system  that 
supports  the  improvement  of  an  organization’s  software  acquisition  process  capability  and 
performance.  The  framework  integrates  an  acquisition-process  reference  model,  such  as  the 
Software  Acquisition  Capability  Maturity  ModelSM  (SA-CMM® );  a  process  that  defines  the 
improvement  approach,  such  as  the  SEI’s  IDEALSM  method;  plus  guidance  and  other 
artifacts,  which  support  the  use  of  the  model  and  improvement  process.  The  guidance  and 
artifacts  are  stored  in  a  repository  that  automatically  links  them  to  the  rest  of  the  framework. 
This  linking  is  structured  to  ensure  that  the  reference  model,  the  improvement  process,  and 
the  supporting  artifacts  are  available  to  the  organization  at  the  right  time  in  the  improvement 
process  phases  and  to  focus  on  the  areas  for  which  the  organization  seeks  improvement.  This 
document  discusses  rationale  behind  the  need  for  the  SAIF,  the  elements  constituting  the 
SAIF,  and  the  intended  operational  usage  of  the  SAIF. 


*  CMM  is  registered  in  the  U.S.  Patent  and  Trademark  Office. 

SM  Capability  Maturity  Model  and  IDEAL  are  service  marks  of  Carnegie  Mellon  University. 
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1.  Introduction 


This  section  presents  an  overview  of  the  current  software  practice  in  the  Department  of 
Defense  (DoD)  software  acquisition  environment,  a  framework  to  improve  system  and 
software  acquisition  processes,  and  an  overview  of  the  Software  Acquisition  Improvement 
Framework  (SAIF)  architecture. 

1.1  Current  Software  Acquisition  Practice 

In  the  Department  of  Defense  (DOD)  acquisition  environment,  most  software  is  acquired  and 
developed  as  one  of  many  components  of  a  larger  system.  Typically,  the  software  acquisition 
follows  the  guidelines  and  framework  of  the  system  acquisition.  In  this  context,  the  software 
acquisition  process,  and  any  improvement  to  this  process,  may  be  suppressed  by  the 
requirement  to  address  the  system-related  issues.  This  suppression,  in  turn,  can  lead  to  poor 
management  of  that  portion  of  the  acquisition  related  to  software. 

No  two  acquisition  programs  are  identical.  Each  program  has  unique  aspects  relating  to 
technology,  cost,  schedule,  priority,  and  funding.  However,  there  is  usually  a  common  life 
cycle  and  acquisition  pattern  underlying  all  major  DOD  and  industry  programs. 

Reform  of  the  federal  acquisition  process  is  now  a  reality.  Public  Law  103-355,  commonly 
referred  to  as  the  Federal  Acquisition  Streamlining  Act  of  1994,  was  established  to  facilitate 
a  more  equitable  balance  between  government-unique  requirements  and  the  need  to  lower 
the  government’s  cost  of  doing  business.  While  the  Act  was  being  formulated,  the  Office  of 
the  Secretary  of  Defense  (OSD)  charted  the  Defense  Science  Board  (DSB)  Task  Force  on 
Acquiring  Defense  Software  Commercially.  The  objectives  of  the  DSB  were  to  do  the 
following: 

•  Determine  conditions  under  which  procurement  of  defense  software  could  include 
commercial  practices  and  determine  the  changes  to  the  Federal  Acquisition  Regulations 
(FARs)  that  would  be  required  to  permit  such  inclusion. 

•  Develop  an  acquisition  strategy  that  incorporates  commercial  practices. 

•  Compare  the  proposed  strategy  with  the  current  DOD  strategy. 

•  Define  institutional  mechanisms  to  assure  that  the  recommendations  are  implemented. 

So  many  far-reaching  recommendations  emerged  from  the  study  that  the  Office  of  the 
Secretary  of  Defense  Acquisition  and  Technology/C31  (OSDA&T/C3I)  chartered  a  Software 
Management  Review  Council  (SMRC)  to  assure  that  these  recommendations  were 
successfully  implemented.  The  SMRC  had  the  support  of  the  OSD-chartered  Software  Best 
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Practices  initiative.  This  initiative  focused  on  improving  and  restructuring  the  way  DOD 
manages  the  acquisition  of  software. 

Despite  the  subsequent  implementation  of  some  of  these  recommendations,  many  defense 
programs  still  do  not  meet  their  objectives  because  of  the  software  inherent  to  their  systems. 
A  high  percentage  of  the  time,  software  is  delivered  late  and  fails  to  meet  the  user’s  needs. 
Extensive  cost  overruns  and  schedule  slips  are  commonplace  due,  in  large  part,  to  software’s 
lack  of  visibility  in  the  larger  system  acquisition  process.  The  software  acquisition  process 
typically  cannot  accommodate  changes  in  requirements,  or  inadequate  estimates  of  the  task 
size  and  the  time  required  to  develop  the  product.  Even  if  changing  requirements  and 
inadequate  estimates  were  not  an  issue,  the  program  manager  (PM)  typically  lacks 
institutionalized  and  measurable  processes,  practices,  methods,  and  techniques  that  support 
successful  management  of  software  acquisition. 

Acquisition  organizations,  in  general,  must  improve  their  software-engineering  and 
acquisition  processes  and  the  practices  that  implement  these  processes.  Project  managers  and 
other  acquisition  managers  need  ways  to  improve  their  software  acquisition  processes  that 
take  advantage  of  Acquisition  Reform  policies  and  find  effective  practices  to  implement 
these  processes.  The  federal  government  and  industry  agree  that  the  acquisition  community 
would  benefit  from  improved  software  acquisition  processes.  A  framework  would  provide  a 
vehicle  to  effect  these  improvements. 

1.2  Approach  for  Acquisition  Process  Improvement 

Process  improvement  for  developers  of  software-intensive  systems  has  taken  on  multiple 
dimensions  as  the  technologies  for  altering  the  software-development  process  evolve  and 
multiply.  In  addition  to  a  reference  model,  such  as  the  Software  Capability  Maturity  Model 
(SW-CMM)  or  ISO  9000,  the  developer  must  choose  or  create  a  process  for  the  improvement 
effort  itself  [Paulk  93a,  Paulk  93b,  ISO  9000].  Popular  process-improvement  models,  such  as 
Shewart's  Plan  Do  Check  Act  (PDCA)  model  or  the  Software  Engineering  Institute's  IDEAL 
model,  may  be  used  or  adapted  to  form  the  basis  for  improvement  effort  activities  [Deming 
86,  McFeeley  96],  The  reference  model  and  process  improvement  model  must  be  brought 
together  and  then  integrated  with  artifacts  such  as  guidance  documents  and  how-to  practices 
that  support  implementing  the  improvement  approach. 

This  approach  of  combining  the  three  dimensions  of  process-improvement  (reference  model, 
improvement  process,  and  supporting  artifacts)  can  be  extended  to  improve  any  process.  In 
this  way,  the  approach  can  be  directly  applied  to  improving  software  acquisition  processes. 

Traditionally,  implementing  such  an  approach  has  been  done  as  an  on-the-fly  activity  by  the 
process-improvement  practitioners  charged  with  upgrading  the  organization’s  processes.  A 
need  exists  for  a  more  rigorous  and  integrated  approach  to  process  improvement,  especially 
for  software  acquisition  process  improvement.  This  approach  must  also  include  a  roadmap  to 
say  what,  when,  and  how  to  improve.  It  is  the  intent  of  the  Software  Acquisition 
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Improvement  Framework  to  integrate  all  of  those  facets  into  a  coherent  approach  for 
acquisition  process  improvement. 

1 .3  Overview  of  the  Software  Acquisition 
Improvement  Framework  (SAIF) 

In  the  following  subsections,  the  Software  Acquisition  Improvement  Framework  (SAIF)  is 
described  in  terms  of  its  architecture,  operational  use,  and  potential  uses. 

1.3.1  SAIF  Architecture 

The  SAIF  is  a  computer-aided  system  that  ties  together  a  process  reference  model,  a 
disciplined  improvement  process,  and  a  repository  that  contains  the  guidance  and  artifacts 
supporting  the  use  of  the  reference  model  and  improvement  process.  The  SAIF  architecture 
consists  of  these  three  elements,  linked  together  to  be  mutually  supportive. 

The  architecture  of  the  SAIF  has  been  generalized  to  allow  it  to  accommodate  alternative 
process-reference  models  and  alternative  improvement  processes,  while  it  maintains  a 
consistent  view  for  the  user.  The  SAIF  ultimately  is  intended  to  be  a  computer-based  tool  set 
that  supports  the  work  of  acquisition  professionals;  this  generalized  architecture  allows  the 
SAIF  to  evolve  from  a  paper-based  to  a  computer-based  form.  The  architecture  also  allows 
the  SAIF  to  accommodate  reference  models  and  improvement  processes  outside  of  the 
acquisition  domain.  For  example,  the  SAIF  could  employ  software  risk  evaluations  or 
domain  analyses  as  a  basis  for  reference  models. 

1 .3.2  Operational  Use 

The  SAIF  presents  an  organization  with  guidance,  methods,  “how-to”  practices,  and  other 
artifacts  from  a  repository  (technology  catalog).  These  methods,  practices,  and  other  artifacts 
guide  the  organisation  through  the  steps  of  process  improvement  and  allow  the  organization 
to  select  and  adapt  the  practices  that  will  improve  the  process. 

The  guidance  and  artifacts  used  are  those  appropriate  to  the  current  phase  of  the 
improvement  process1  and  those  appropriate  to  the  areas  of  the  reference  model  that  allow 
the  organization  to  plan  and  implement  improvements.  For  example,  if  the  organization 
needs  to  improve  a  particular  area  of  its  acquisition  process,  such  as  solicitation  planning, 
and  execution,  the  SAIF  presents,  from  the  repository  (technology  catalog),  the  guidance  and 
alternative  how-to  practices  appropriate  to  that  area. 


1  See  the  IDEAL  model  for  definitions  of  phases  for  the  improvement  process. 
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1.3.3  Uses 

We  anticipate  that  an  organization  will  use  the  SAIF  to  do  the  following: 


•  Guide  the  organization’s  improvement  process  to  a  particlar  maturity  level  of 
improvement.2 

•  Diagnose  process  capability  and  performance. 

•  Analyze  where  improvements  are  needed. 

•  Set  goals  and  priorities  for  the  improvement  activities. 

•  Define  process  changes  or  additions  to  process  capabilities. 

•  Identify  which  technologies  are  needed  for  improvement. 

•  Plan  the  insertion  of  technologies  into  the  organization,  implement  this  insertion,  and 
obtain  feedback  on  the  results. 


2  See  CMM  definition  for  maturity  level. 
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2.  SAIF  Conceptual  Architecture  Definition 


This  section  defines  the  SAIF,  its  generic  architecture,  and  the  planned  initial 
implementation  of  this  architecture.  The  definition  of  the  SAIF  and  its  architecture  are 
derived  from  the  general  operational  requirements  described  in  Appendix  A. 

2.1  Definition 

The  Software  Acquisition  Improvement  Framework  is  an  extensible  system  consisting  of  a 
disciplined  process  for  improvement,  a  reference  model,  and  supporting  artifacts,  all  of 
which  are  integrated  to  be  mutually  supportive  and  to  facilitate  the  improvement  of  an 
acquisition  process. 

2.2  Generic  Architecture 

Figure  1  shows  the  three  components  of  the  framework:  process  for  improvement,  reference 
model,  and  repository  (technology  catalog). 

2.2.1  Reference  Model 

The  reference  model  component  of  the  framework  allows  the  user  to  choose  among  reference 
models,  which  will  be  used  to  measure  process-improvement  success.  Every  process- 
improvement  effort  must  include  a  standard  that  can  be  used  to  determine  the  effort’s 
status — a  diagnostic  reference  that  characterizes  the  quality  of  the  process  to  be  improved 
and  that  can  be  used  to  measure  the  progress  of  the  improvement.  The  SAIF  architecture 
allows  for  a  variety  of  reference  models  that  can  provide  the  characteristics  of  desired 
practices.  The  reference  model  answers  the  question,  “Where  am  I?”  by  allowing  a 
comparison  between  the  organization’s  acquisition  processes  and  those  defined  in  the 
reference  model.  The  reference  model  also  answers  the  question,  “Where  do  I  need  to  go?” 
by  defining  the  characteristics  of  desired  processes,  which  may  be  used  as  goals  for  a 
process-improvement  effort. 

2.2.2  Process  for  Improvement 

The  process  for  improvement  component  of  the  framework  allows  the  user  to  choose  among 
process-improvement  approaches.  The  SAIF  architecture  allows  for  a  process  that  defines 
the  set  of  steps  and  activities  to  be  performed  in  the  improvement  effort.  This  process 
answers  the  question,  “When  and  what  route  do  I  take  to  improve?”  by  defining  the  steps  to 
be  accomplished  in  improvement  and  the  activities  within  each  step.  Alternate  improvement 
processes  are  discussed  in  Appendix  B. 
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Figure  1:  SAIF  Architecture 


2.2.3  Technology  Catalog 

The  technology  catalog  component  of  the  framework  provides  technology,  in  the  form  of 
both  guidance  and  practices,  to  the  user  in  support  of  process  improvement  in  the 
organization.  Every  process-improvement  effort  must  have  the  appropriate  technical 
practices  (“how  to’s”)  ready  for  insertion  into  the  improvement  effort  at  the  appropriate  time. 
The  technology  catalog  is  a  repository  of  guidance  and  practices  for  process  improvement, 
organized  by  the  categories  and  steps  of  the  reference  model  and  the  improvement  process. 

The  artifacts  or  supporting  technologies  aid  the  application  of  the  framework  to  the 
improvement  effort.  These  artifacts  and  technologies  include  both  processes  and  products 
(e.g.,  generic  templates,  guidance,  and  training  packages).  Templates  include  generic  plans 
and  policies  that  must  be  tailored  to  a  specific  environment  to  support  improvement. 

Through  these  artifacts,  the  framework  provides  a  mechanism  for  addressing  the  needs  of  the 
software  acquisition  community  by  disseminating  and  sharing  guidance,  experience, 
knowledge,  and  artifacts  related  to  technology  adoption.  The  technology  catalog  answers  the 
important  question,  “How  have  others  done  this?”  by  cataloging  practices  based  on 
experience.  By  providing  alternative  practices  within  practice  areas,  the  technology  catalog 
provides  the  user  with  options  that  can  be  chosen  to  optimize  the  process-improvement  effort 
for  the  organization,  the  culture,  and  the  work. 
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2.3  Planned  Implementation 

The  SAIF  architecture  is  initially  instantiated  with  the  following  elements: 

2.3.1  The  Software  Acquisition  Capability  Maturity  Model  (SA- 
CMM) 

Figure  2  illustrates  the  SA-CMM  as  the  improvement  reference  model.  The  SA-CMM  is  a 
staged  capability  maturity  model  exhibiting  levels  of  maturity  that  an  organization  can 
achieve  by  effectively  implementing  processes  to  accomplish  the  activities  and  common 
features  described  in  the  model.  In  terms  of  the  SAIF,  the  model  is  used  as  a  basis  to 
determine  the  current  process  capability  of  an  organization  relative  to  acquisition  and  to 
establish  a  set  of  goals  for  improvement.  The  model  depicted  in  Figure  2  encompasses  5 
maturity  levels  and  16  key  process  areas  (KPAs).  As  the  maturity  level  increases,  the  quality 
of  the  process  is  improved.  For  a  further  description  of  the  SA-CMM  refer  to  CMU/SEI-96- 
TR-020  [Ferguson  96]. 


Level 

Focus 

Key  Process  Areas 

5 

Optimizing 

Continuous 

process 

improvement 

Acquisition  Innovation  Management 
Continuous  Process  Improvement 

Quality 

Productivity 

4 

Quantitative 

Quantitative 

management 

Quantitative  Acquisition  Management 
Quantitative  Process  Management 

3 

Defined 

Process 

Standardization 

Training  Program 

Acquisition  Risk  Management 

Contract  Performance  Management 
Project  Performance  Management 
Process  Definition  and  Maintenance 
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2 

Repeatable 

Basic 

Project 

Management 

Transition  to  Support 

Evaluation 

Contract  Tracking  and  Oversight 

Project  Management 

Requirements  Development  and  Mgmt. 
Solicitation 

Software  Acquisition  Planning 

: : '  •'  >y  ir.'  ' :  i 

Risk 

1 

Initial 

Competent  people  and  heroics 

Rework 

Figure  2:  Software  Acquisition  Capability  Maturity  Model 


2.3.2  The  IDEAL  Model 

The  IDEAL  model  serves  as  the  improvement  process  for  the  first  instance  of  the  SAIF  and 
is  mapped  into  acquisition-specific  steps  for  inclusion  in  the  SAIF  technology  catalog.  The 
IDEAL  model  shown  in  Figure  3  consists  of  five  phases  organized  to  provide  a  structured 
approach  to  process  improvement  and  adoption  of  technologies  to  support  process 
improvement.  The  continuous  loop  through  the  steps  is  an  indication  that  this  process  can  be, 
and  usually  is,  a  continual  process.  The  length  of  time  it  takes  to  complete  a  cycle  through 
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the  model  will  vary  from  organization  to  organization,  the  improvements  planned,  and  the 
resources  available.  Depending  on  the  resources,  many  of  the  steps  or  activities  may  occur  in 
parallel.  In  addition  the  model  can  be  tailored  to  each  organization’s  environment  and 
constraints  for  process  improvement. 


Learning 


Stimulus  for  Change 


Acting 


Diagnosing 


Establishing 


Figure  3:  The  IDEAL  Improvement  Process 

2.3.3  The  Technology  Catalog 

The  Software  Acquisition  Technology  Catalog  contains  the  both  the  artifacts,  or  references 
to  artifacts  (such  as  generic  templates  and  other  technologies)  that  support  the 
implementation  of  IDEAL  and  those  technologies  that  support  process  definition  and 
improvement  against  the  reference  model.  The  catalog  provides  the  “links”  to  the  appropriate 
stages  of  the  IDEAL  model  for  implementation  and  to  the  specific  areas  of  improvement 
identified  by  the  reference  model. 

Where  possible,  each  technology  and  artifact  in  the  catalog  will  include  associated  data  that 
supports  the  technology  or  artifact  from  the  standpoint  of  technology  adoption  or  technology 
insertion  (for  example,  how  was  it  used  in  the  past,  under  what  circumstances,  what  was  the 
return  on  investment,  and  how  did  it  fit  into  the  business  context  of  the  organization).  In 
most  cases,  the  stored  items  in  the  catalog  are  references  to  more  detailed  descriptions  of  the 
technology  to  be  applied.  See  Section  3  for  a  detailed  description  of  the  technology  catalog. 
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3.  Software  Acquisition  Technology 
Catalog 


This  section  describes  the  Software  Acquisition  Technology  Catalog  (catalog)  and  how  it  is 
employed  with  the  process  and  reference  models  of  the  framework. 

3.1  Catalog  Notional  Structure 

The  Software  Acquisition  Technology  Catalog  is  the  pivotal  component  of  the  framework.  It 
is  a  computer-aided,  extensible  repository.  As  depicted  in  Figure  4,  this  repository  consists  of 
a  database  and  a  database  management  system,  which  is  accessed  through  a  graphical  user 
interface. 


Figure  4:  SAIF  Notional  Structure 
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The  repository  stores  information  (artifacts)  that  supports 


•  implementation  and  instantiation  of  a  process  improvement  model  such  as  IDEAL 

•  application  of  a  reference  model  such  as  the  SA-CMM 

•  other  artifacts  (e.g.,  guidance,  “how-to”  practices,  and  tools)  that  an  organization  may 
adapt  to  improve  its  software  acquisition  process 

As  new  technologies  that  support  the  framework  mature,  these  technologies  are  added  to  the 
repository  as  artifacts.  In  many  cases,  the  stored  artifacts  are  references  to  the  more  detailed 
description  of  the  technology  to  be  applied,  along  with  an  abstract  of  the  technology. 

Another  perspective  of  the  repository  notional  structure  is  shown  in  Figure  5,  which 
illustrates  how  artifacts  are  categorized  within  the  database.  The  artifacts  may  be  stored  by 

•  areas  within  the  improvement  reference  model 

•  steps/sub-steps  in  the  improvement  process 

•  level  of  information  — either  guidance,  practices,  or  applicable  tools 

Figure  5  uses  example  categories  from  the  SA-CMM  as  a  reference  model,  and  IDEAL  as  a 
process  improvement  model.  In  practice,  the  categories  will  vary  with  the  selected  reference 
model  and  improvement  process. 

3.2  Catalog  Levels  of  Information 

The  artifact  information  in  the  SAIF  exists  at  three  levels  of  abstraction:  guidance,  practices, 
and  tools.  While  the  demarcation  line  between  these  levels  of  information  abstraction  is  not  a 
hard-and-fast-boundary,  the  SAIF  uses  the  following  operational  definitions: 

Guidance:  Information  that  tells  the  user  what  needs  to  be  done  and  when  it  needs  to  be 
done.  Guidance  is  at  a  higher  level  of  abstraction  than  practices,  and  often  establishes  the 
need  for  practices  by  stating  that  the  user  should  be  doing  an  activity,  without  providing  the 
details  of  how  to  do  the  activity.  For  example,  guidance  might  state  that  the  user  should  have 
a  plan  for  risk  management  and  that  the  plan  should  cover  certain  specified  aspects  of  risk 
management. 

Practice:  Information  that  tells  the  user  how  to  do  an  activity  or  set  of  activities  within  a 
process.  Practices  are  more  detailed  than  guidance  and  provide  steps  that  a  user  can  follow  to 
accomplish  the  specified  activity.  An  example  of  a  practice  might  be  a  template  plan  for  risk 
management,  accompanied  by  tailoring  guidelines. 
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3.3  Artifact  Information 

In  order  for  the  SAIF  to  accomplish  its  goals,  a  consistent  set  of  data  for  each  artifact  must 
be  stored  in  the  repository.  The  consistent  set  of  data  is  captured  in  the  criteria  shown  in 
Figure  6.  These  criteria  are  used  to  analyze  the  artifacts  for  inclusion  in  the  repository  and 
provide  sufficient  information  for  the  SAIF  to  be  used.  Detailed  explanation  of  these  criteria 
is  given  in  Appendix  C. 

3.4  Repository  Linkages 

For  the  initial  instantiation  of  the  SAIF,  the  catalog  provides  the  “linking”  of  all  artifacts 
residing  in  the  catalog  to  the  appropriate  place  within  the  IDEAL  cycle  and  to  specific  key 
process  areas  (KPAs)  of  the  SA-CMM.  Properly  employed  by  an  acquisition  organization,  its 
intended  use  is  to  identify  those  artifacts  that  are  needed  to  implement  the  reference  model, 
support  implementation  of  the  improvement  process,  and  provide  the  artifacts  to  support  the 
organization’s  process  improvement. 

Table  1  illustrates  a  small  sample  of  the  data  stored  and  shows  how  these  technologies  are 
related  to  the  phases  of  the  IDEAL  model  and  KPAs  of  the  SA-CMM.  (The  table  does  not 
indicate  the  actual  data  structures  or  schema  used  in  the  repository.) 
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Name  of  practice _ 

Description  of  practice _ 

Purpose _ 

Process  areas 

Applicable  common  features 
Acquisition  life-cycle  phase 
Resources 


Usage/limitations _ 

Pros _ 

Cons _ 

Origin  (optional) _ 

Supporting  tools,  techniques,  and  technologies 

Recommendations  for  change _ 

Reference,  information  sources,  point  of  contact 
Last  modified 


Acceptance/Quality  Criteria  _ 

Software-intensive  systems  acquisition  domain 
Legal  and  regulatory  compliance 
Well  documented 

Maturity/successful  application  history 
Overall  return  on  investment 

_ External  reviewers/evaluators _ 

Figure  6:  Minimum  Descriptive  Information  and  Criteria  for  SAIF  Artifacts 

3.5  Catalog  Query  Program 

In  addition  to  storing  artifacts,  the  catalog  includes  a  second  component — an  interactive 
query  program  that  guides  users  through  the  framework.  By  answering  questions,  the  users 
can  focus  on  appropriate  artifacts  to  support  their  improvement  needs. 

Based  on  use  of  the  IDEAL  and  SA-CMM  instantiation  of  the  SAIF,  the  operational  use  of 
the  catalog  would  be  to  query  the  repository,  “sorting”  on  the  phase  of  the  improvement 
process  and  reference  model  links  to  specific  key  practices  and  whether  guidance,  practices, 
or  tools  were  desired.  The  response  would  provide  the  user  a  listing  of  all  the  information 
sources  that  satisfy  the  query  by  title.  Subsequent  query  of  these  sources  would  provide  the 
user  information  stored  for  that  artifact  such  as  the  information  shown  in  Figure  6  above. 
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SAIF  Improvement  Process  Phase 
Artifacts  (Technologies) _ 


Links  to  SA-CMM  KPAs 


■ 

■ 

Process  models 

M 

■ 

IDEAL  technical  reference  model: 
CMU/SEI-96-HB-001 

B 

B 

X 

IDEAL  implementation  guidelines 
and  templates 

X 

X 

B 

X 

X 

Reference  model 

SA-CMM  (CMU/SEI-96-TR-020) 

n 

H 

SA-CMM  appraisal  process 

X 

SA-CMM  familiarization  kit 

H 

SA-CMM  Maturity  Questionnaire 

n 

Tech  transition 
training  &  assistance 

■ 

1 

■ 

■ 

SEI  Open  Systems 

■ 

■ 

Intro,  to  SA-CMM 

CBA  lead  assessor 

SA-CMM  policy  templates 

■ 

B 

All  SA-CMM  KPAs 

SA-CMM  planning  templates 

x 

All  SA-CMM  KPAs 

X 

. 

Team  risk  management 

B 

19 

Software  Risk  Evaluations 

E 

B 

IB9 

Table  1:  Notional  Concept  of  Technology  Catalog  Linkages 


Queries  are  effected  through  a  windowing,  graphical  user  interface  that  transparently 
provides  the  user  the  necessary  search  (query)  mechanism  to  access  the  data. 

Figure  7  provides  a  notional  concept  of  operations,  which  will  be  discussed  in  Section  4. 
Here  the  user  requests  information  on  guidance  and  practices.  The  technology  catalog 
provides  a  perspective  from  the  reference  model  or  from  the  improvement  process  of  the 
artifacts  that  support  these  points  of  view. 
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q  Guidance 
Q'  Practices 
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User 

determines 
level  of 
assistance 
desired 


User  selects 
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reference 
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improvement 
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interest 
(Example: 
Diagnosing: 
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Current  & 
Desired  State) 
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Management 
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Figure  7:  Use  of  the  Automated  Technology  Catalog 
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4.  Intended  Operational  Use  of  the  SAIF 


This  section  describes  how  the  Software  Acquisition  Improvement  Framework  (SAIF)  is 
intended  to  be  applied,  by  stepping  through  the  phases  of  the  IDEAL  model,  as  an  example, 
and  indicating  at  what  points  the  reference  model  and  technology  catalog  come  into  play. 

To  apply  the  framework,  the  acquisition  organization  follows  the  steps  described  in  the 
IDEAL  model,  employing  appropriate  artifacts  from  the  catalog.  Because  IDEAL  is  a  general 
model,  the  organization  may  need  to  tailor  its  activities  and  scheduling  of  the  steps  to  its 
specific  environment  or  circumstances.  (For  example,  an  organization  may  have  already  been 
diagnosed  and  know  areas  for  software  acquisition  improvement.  In  this  case,  the  initial  and 
diagnostic  phases  of  the  IDEAL  model  may  be  passed  over.)  Here,  the  supporting  artifacts 
from  the  technology  catalog  are  the  description  of  the  IDEAL  model  and  tailoring  guidelines 
to  alter  activities  and  procedures  for  each  phase  of  this  model.  The  catalog  indicates  these 
items  linked  to  this  tailoring.  Following  this  initial  tailoring,  the  organization  proceeds 
through  the  appropriate  IDEAL  phases  as  described  below. 

4.1  Initiating  Phase 

The  initiating  phase  is  the  starting  point  of  the  process.  This  is  where  the  stimulus  for 
improvement  is  generated.  This  stimulus  could  take  many  forms,  but  typically,  it  occurs 
when  senior  management  first  understands  the  need  for  improving  the  acquisition  process. 
This  phase  also  is  where  the  context  of  the  improvements  is  documented,  derived  from  the 
business  or  enterprise  goals  of  the  organization.  The  initial  infrastructure  to  conduct  the 
improvement  program  is  established,  the  role  and  responsibilities  for  the  infrastructure  are 
defined,  and  initial  resources  are  assigned.  This  latter  statement  implies  that  there  is  a 
commitment  from  senior  management  to  improve  its  software  acquisition  process — a  crucial 
requirement  for  beginning  this  process.  An  initial  plan  is  developed  to  baseline  the  need  for 
improvement  and  set  up  the  infrastructure. 

Example  artifacts  that  support  the  IDEAL  implementation  in  this  phase  are  the  IDEAL  model 
technical  reference,  the  detailed  IDEAL  procedures  to  be  followed  in  this  phase,  and  a 
template  for  the  plan. 

Artifacts  that  support  the  instantiation  of  IDEAL  for  acquisition  include  the  SA-CMM  and 
its  supporting  artifacts,  which  include  the  SA-CMM  familiarization  kit  and  SA-CMM 
application  guidelines. 
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4.2  Diagnosing  Phase 

In  the  diagnosing  phase,  the  organization  starts  on  the  path  to  continuous  improvement  of  it 
software  acquisition  process.  This  phase  lays  the  ground  work  for  the  remaining  phases.  In 
this  phase,  appraisal  activities  are  usually  performed  to  establish  the  organization’s  process 
maturity  baseline.  (The  maturity  baseline  is  the  result  of  using  the  SA-CMM  as  the  reference 
model.)  The  results  and  recommendations  from  the  appraisals  and  any  other  baselining 
activities  will  be  reconciled  with  existing  and/or  planned  improvement  efforts  for  inclusion 
into  an  acquisition  process  improvement  plan,  (see  Section  4.3,  Establishing  Phase). 

An  artifact  that  supports  the  IDEAL  implementation  in  this  phase  is  the  IDEAL  model 
technical  reference  [McFeeley  96]. 

Artifacts  that  support  the  instantiation  of  IDEAL  for  acquisition  include  the  SA-CMM  and 
its  supporting  products,  which  include  the  following: 

•  SA-CMM  application  guidelines 

•  Appraisal  methodology  associated  with  the  SA-CMM  (CMM-Based  Appraisal  for 
Internal  Process  Improvement  [CBA IPI]) 

•  Introduction  to  the  SA-CMM  training  course 

•  Lead  assessor  training  course  for  CBA  IPI 

Results  of  an  SA-CMM  appraisal  are  provided  to  senior  management  to  gamer  or  sustain 
their  support  to  continue  with  the  improvement  initiative.  The  documented  results  would  also 
be  additional  input  to  the  improvement  plan  initially  developed  during  the  initiating  phase. 

4.3  Establishing  Phase 

During  this  phase,  an  improvement  plan,  which  is  a  continuation  of  the  plan  developed 
during  the  initiating  phase,  is  generated  in  accordance  with  the  organization’s  vision, 
enterprise  goals,  strategic  plan,  and  lessons  learned  from  past  improvement  efforts.  Senior 
management  considers  the  areas  that  need  to  be  improved  and  decides  which  areas  to 
implement,  again  based  upon  improvement  needs,  available  resources,  and  other 
organizational  policies  and  constraints. 

The  improvement  areas  that  the  organization  has  decided  to  address  with  its  improvement 
efforts  are  prioritized,  and  strategies  for  pursuing  the  solutions  are  developed.  The  draft  of 
the  action  plan  will  be  completed  in  accordance  with  the  organization’s  vision,  strategic 
plans,  lessons  learned  from  past  improvement  efforts,  key  business  issues,  and  long-range 
goals.  During  this  phase,  measurable  goals  or  objectives  for  the  improvement  effort  are 
developed  from  the  general  goals  developed  in  the  initiating  phase.  Measurements  will  be 
developed  to  monitor  progress  of  the  improvement  initiative.  Finally,  resources  are 
committed  and  training  necessary  to  carry  out  the  planned  improvement  effort  is 
accomplished. 
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In  this  planning  effort,  process-definition  activities  are  carried  out  so  that  solutions  can  be 
applied  to  effect  the  necessary  improvement.  The  supporting  artifacts  play  heavily  in  this 
phase.  This  phase  matches  results  from  the  diagnosing  phase  with  technologies  and  artifacts 
that  are  believed  to  improve  the  overall  acquisition  process.  In  order  to  improve  with  respect 
to  this  model,  the  acquisition  organization  must  modify  or  put  into  place  a  process,  a  set  of 
processes,  or  other  technologies  and  artifacts,  to  accomplish  those  activities  and  key  features 
of  the  reference  model,  which  had  been  identified  as  areas  for  improvement. 

Using  the  SA-CMM  as  a  reference,  improvement  is  not  simply  a  matter  of  introducing  or 
modifying  processes  that  accomplish  all  the  activities  or  key  features  for  that  KPA.  It  may 
appear  that  the  way  to  improvement  is  introducing  the  processes  needed  to  master  all  the 
KPAs  at  one  level  and  be  in  the  range  of  the  next  level.  Thus,  if  an  acquisition  organization 
masters  all  the  KPAs  at  Level  2,  then  it  should  be  at  the  beginning  range  of  Level  3. 

However,  it  is  not  simple.  The  process  or  processes  to  be  introduced  must  be  considered  in 
terms  of  value  added,  return  on  investment,  and  long-range  goals  before  they  are  deployed. 
The  selection  of  a  new  technology  must  also  take  into  account  any  constraints  facing  the 
acquisition  organization  and,  possibly,  each  project  within  that  organization.  The  point  is  that 
introducing  a  technology — a  new  process,  for  example — for  the  sake  of  being  rated  higher 
against  the  SA-CMM  reference,  may  not  yield  an  improvement  in  the  organization’s 
performance.  All  aspects  of  introducing  a  technology  must  be  considered,  and  the  technology 
must  be  tailored  to  the  organization’s  needs  before  introduction  in  order  for  the  technology 
to  be  introduced  successfully. 

Artifacts  that  support  the  IDEAL  implementation  in  this  phase  are  the  IDEAL  model 
technical  reference  [McFeeley  96]  and  a  template  for  the  plan. 

Artifacts  that  support  the  instantiation  of  IDEAL  for  acquisition  include  the  SA-CMM  , 
implementation  guidelines  for  the  SA-CMM  (which  includes  templates  for  policies  and  plans 
called  for  in  the  SA-CMM),  and  all  the  “how-to”  technologies  in  the  catalog  that  are 
applicable  to  the  planned  areas  of  improvement. 

For  the  “how-to”  technologies,  the  catalog  will,  in  most  cases,  reference  the  technology  and 
its  supporting  artifacts.  These  “how-to”  technologies  are  not  necessarily  focused  on  specific 
or  distinct  areas  that  may  be  identified  by  the  SA-CMM;  therefore,  the  “links”  shown  in  the 
catalog  will  indicate  the  application  of  a  technology  to  more  than  one  area  of  improvement. 
Thus,  the  process  designer  must  understand  and  apply  these  technologies  to  fit  the  needs  of 
the  organization  (see  the  Section  4.4,  Acting  Phase). 

4.4  Acting  Phase 

In  the  acting  phase,  solutions  to  address  the  areas  of  improvement  defined  in  the  preceding 
phases  are  developed  and  piloted.  Lessons  learned  are  collected,  in  addition  to  metrics  on 
performance  and  goal  achievement.  These  activities  of  collecting  information  on  solutions 
are  not  confined  to  the  solutions  identified  for  improvement  in  previous  phases.  Past 
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improvement  efforts  may  already  be  in  place,  and  projects  may  have  applied  other 
acquisition  techniques  to  their  specific  project.  Information  is  collected  on  these  efforts,  as 
well. 

In  terms  of  the  technologies  and  artifacts  proposed  in  the  establishing  phase,  the  acting  phase 
creates  or  “tailors”  those  proposed  solutions  to  satisfy  the  organization’s  needs.  Some  of  the 
“how-to”  technology  artifacts  will  have  associated  guidance  to  help  the  designer  tailor  or 
otherwise  adapt  them  to  the  needs  of  the  organization. 

An  artifact  that  supports  the  IDEAL  implementation  in  this  phase  is  the  IDEAL  model 
technical  reference. 

4.5  Learning  Phase 

The  objective  of  the  learning  phase  is  twofold.  First,  if  solutions  developed  during  the  acting 
phase  were  successful  or  show  promise  for  an  improved  acquisition  process,  these  are 
included  in  the  organization’s  standard  software  acquisition  process  and  are  made  available 
for  use  throughout  the  acquisition  organization.  Second,  artifacts  such  as  lessons  learned  and 
metrics  on  performance  and  goal  achievement  have  been  collected  to  make  the  next  pass 
through  the  IDEAL  model  more  effective.  By  this  time,  solutions  have  been  developed. 

These  are  added  to  the  organization’s  repository,  which  will  become  the  source  of  data  for 
the  next  pass  through  the  IDEAL  model. 
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5.  Summary 


This  section  summarizes  the  concepts  and  describes  implementation  plans  for  the  SAIF. 

5.1  Definition  and  Planned  Implementation 

The  SAIF  is  an  extensible  framework  that  assists  organizations  in  improving  their  software 
acquisition  capability.  The  framework  consists  of  three  elements: 

•  reference  model  for  improvement-  allows  a  choice  among  reference  models  against 
which  process  improvement  can  be  measured 

•  process  for  improvement  -  allows  a  choice  among  process  improvement  approaches 

•  technology  catalog  -  provides  technology  in  the  form  of  both  guidance  and  practices,  to 
support  an  organization  in  process  improvement 

Initially  the  implementation  will  use  the  following  models: 

•  The  Software  Acquisition  Capability  Maturity  Model  (SA-CMM)  will  be  used  as  a  basis 
to  determine  the  current  capability  of  an  organization  relative  to  acquisition  processes 
and  to  establish  a  set  of  goals  for  improvement. 

•  The  IDEAL  model  will  serve  as  the  improvement  process  for  the  SAIF. 

•  The  Software  Acquisition  Technology  Catalog  contains  the  artifacts,  or  references  to 
artifacts,  such  as  generic  templates  and  other  technologies,  that  support  both  the  IDEAL 
model  and  the  SA-CMM.  The  catalog  provides  the  “links”  to  the  appropriate  stages  of 
the  IDEAL  model  for  implementation  of  this  model  and  to  the  specific  areas  of 
improvement  that  are  identified  by  the  reference  model. 

5.2  Future  Work 

Several  efforts  are  needed  to  complete  the  SAIF  implementation. 

First,  characteristics  of  the  guidance  and  effective  practices  needed  to  support  the  IDEAL 
model  and  the  SA-CMM  must  be  defined.  This  characterization  will  allow  the  development 
of  such  artifacts  or  the  elicitation  of  such  artifacts  for  the  inclusion  into  the  technology 
catalog.  The  World  Wide  Web  (WWW)  is  one  mechanism  that  may  be  used  to  elicit  such 
artifacts.  Workshops  and  user  needs  analyses  are  planned  to  support  the  elicitation. 


Second,  the  graphical  user  interface  and  the  linking  of  the  artifacts  is  to  be  accomplished 
with  a  database  management  system  (DBMS)  as  part  of  the  technology  catalog.  A  second 
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possibility  is  to  use  the  WWW.  The  decision  on  which  to  use  depends  upon  the  transition 
mechanism  that  would  most  benefit  the  user. 
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Appendix  A:  General  Operational 
Requirements  for  the  SAIF 


Organizations  currently  involved  with  the  acquisition  of  software  and  software-intensive 
systems  do  not  have  the  knowledge,  technology,  or  motivation  to  improve  their  acquisition 
process.  The  purpose  of  the  framework  is  to  provide  a  disciplined  approach  supported  by 
automation  that  helps  organizations  to  improve  their  software  acquisition  process  capability. 

It  is  anticipated  that  an  organization  will  use  the  framework  to 

•  diagnose  its  software  acquisition  process  capability 

•  analyze  where  improvements  are  needed 

•  define  processes  changes  or  additions  to  their  capabilities 

•  set  goals  and  priorities  for  improvement 

•  identify  what  technologies  are  needed  for  improvement 

•  plan  introduction  of  these  technologies  into  the  organization 

•  implement  this  introduction  and  obtain  feedback  on  the  results  of  these  technologies 

General  Requirements 

The  SAIF  will  provide  an  extensible  framework  that  helps  organizations  to  improve  their 
software  acquisition  capability.  (A  framework  is  defined  as  a  basic  arrangement,  form, 
system,  or  set  of  relationships.  A  framework  may  consist  of  data  organized  into  a  structure 
that  gives  insight  into  the  practices  and  technologies  that  may  constitute  the  framework.) 

The  SAIF  will  provide  a  structured  methodology  to  appraise  and  identify  technologies  (both 
practices  and  products)  employed  to  improve  an  organization’s  acquisition  capability.  Table 
2  illustrates  potential  uses  of  the  SAIF.  There  are  no  known  systems  that  currently  provide 
the  structured  methodology  that  is  provided  by  the  SAIF. 
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SAIF  Use 

Description 

1 

Process  guidance 

Help  define  an  organization’s  software 
acquisition  capability  and  use  as  a  motivational 
tool  to  improve  the  process(es)  within  the 
organization 

2 

Process  capability 
appraisal 

Provide  an  approved  methodology  to  support 
capability  appraisals  to  diagnose  the 
organization’s  software  acquisition  process 
capability  relative  to  a  reference  model 

3 

Acquisition  process 
framework 

Provide  a  mechanism  to  apply  technologies  (both 
practices  and  tools)  that  support  and  maintain 
continuous  improvement  and  obtain  feedback  on 
the  results  of  these  technologies 

■ 

Plan  for 
improvement 

Support  planning  the  introduction  of  these 
technologies  into  the  organization 

Table  2:  Employment  of  the  SAIF 


The  SAIF  is  an  overarching  system  that  will  be  the  integration  mechanism  for  software 

acquisition  products  and  technologies.  It  will  focus  these  products  and  technologies  on 

improving  the  organizations’  acquisition  processes. 

Capabilities  Required 

The  SAIF  will 

•  provide  a  structured,  disciplined  process  explaining  step-by-step  procedures  for  an 
organization  to  follow  in  order  to  initiate,  diagnose,  plan,  implement,  learn,  and, 
thereby,  manage  the  adoption  of  technologies  (processes,  products,  tools,  etc.)  to 
improve  its  software  acquisition  process  capability 

•  contain  a  repository  of  guidance  and  specific  artifacts  (or  references  to  artifacts 
[technologies])  that  support  the  disciplined  approach  to  improve  the  software 
acquisition  process 

•  be  sufficiently  flexible  to  accommodate  upgrades  in  technologies  for  the  SAIF  itself 

•  provide  a  user-friendly  interface  in  operation 

•  use  an  established  reference  model  as  a  basis  to  determine  process  capability  and 
process  improvement  opportunities 

•  provide  automation  tools  to  support  the  operation  of  the  framework 
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•  provide  an  automatic  mechanism  to  allow  the  user  to  link  repository  artifacts  that  could 
be  exploited  to  initiate,  diagnose,  plan,  implement,  learn,  and,  thereby,  manage  the 
adoption  of  technologies  to  exploit  specific  improvement  opportunities  identified  by  the 
SAIF 

•  store  or  reference  “how-to”  practices  that  can  be  used  to  implement  the  steps  of  or 
define  the  acquisition  processes  of  an  organization 

•  be  designed  to  accommodate  alternative  reference  models  and  improvement  processes 

•  use  personal  computer  technology  to  implement  the  automated  portion  of  the  systeir 

•  contain  reference  to  technologies  that  support  the  process  improvement 

•  require  no  direct  interface  to  other  systems  other  than  reference  links  to  these 
technologies  stored  in  the  repository 

•  link  technologies  to  appropriate  improvement  areas  within  the  improvement  reference 
model 

•  be  maintained  initially  by  the  Software  Engineering  Institute  (SEI) 

•  include  training  for  operation  of  the  SAIF 
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Appendix  B:  Alternative  Improvement 
Processes 


The  SAIF  architecture  was  developed  to  be  general  in  nature.  It  does  not  presume  a 
particular  reference  model  for  the  software  acquisition  process,  nor  does  it  assume  a 
particular  process  for  improvement.  This  generalization  was  used  to  allow  future  selection  of 
a  reference  model  and  an  improvement  process  from  among  viable  candidates  that  may 
exist.  Allowing  for  alternative  reference  models  and  improvement  process  models  requires 
that  the  SAIF  be  configured  (instantiated)  for  the  chosen  models.  This  would  require  a 
change  to  the  technology  catalog  to  incorporate  the  correct  links  from  the  technologies  stored 
to  the  reference  and  process  models. 

Example  Reference  Models 

For  example,  one  might  select  the  Software  Acquisition  Capability  Maturity  Model  as  the 
reference  model.  (This  model  has  the  structure  shown  in  Figure  8.)  The  implication  is  that 
the  information  in  the  SAIF — guidance  and  practices  that  define  “how-to”  information  for 
improvement — will  then  be  organized  around  the  categories  of  the  reference  model.  Thus  we 
would  see  information  categories  such  as 

•  key  process  areas  (16  defined  areas) 

•  activities  (  96  defined  within  the  16  key  process  areas) 

•  metrics  (one  per  key  process  area) 

•  verification  (two  per  key  process  area) 

A  user  of  the  SAIF  would  access  information  according  to  categories  defined  by  the 
reference  model.  If  he  or  she  were  interested  in  the  project  management  area,  information 
would  be  accessed  by  that  key  process  area  (KPA)  and  by  the  activity,  metric,  or  verification 
practice  within  that  KPA. 
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Level 

Focus 

Key  Process  Areas 

5 

Optimizing 

Continuous 

process 

improvement 

Acquisition  Innovation  Management 
Continuous  Process  Improvement 

Quality 

Productivity 

4 

Quantitative 

Quantitative 

management 

Quantitative  Acquisition  Management 
Quantitative  Process  Management 

■ 

3 

Defined 

Process 

Standardization 

Training  Program 

Acquisition  Risk  Management 

Contract  Performance  Management 
Project  Performance  Management 
Process  Definition  and  Maintenance 

m 

2 

Repeatable 

Basic 

Project 

Management 

Transition  to  Support 

Evaluation 

Contract  Tracking  and  Oversight 

Project  Management 

Requirements  Development  and  Mgmt. 
Solicitation 

Software  Acquisition  Planning 

Risk 

1 

Initial 

Competent  people  and  heroics 

Rework 

Figure  8:  SA-CMM  Structure 

If  the  user  had  a  reference  model  with  seven  practice  areas,  such  as  the  Software  Program 
Managers’  Network  (SPMN’s)  Best  Practices  Framework  [SPMN  98],  the  categories  of 
information  would  be  defined  by  that  model.  Those  categories  would  be 

•  risk  management 

•  requirements  management 

•  planning  and  tracking 

•  quality 

•  interface  management 

•  project  stability 

•  project-unique  issues 

Other  reference  models  that  may  be  used  include  the  System  Engineering  Capability 
Maturity  Model  (SE-CMM),  the  SPICE  (Software  Process  Improvement  Capability 
Determination)  model,  and  the  SDCE  (Software  Development  Capability  Evaluation)  model. 


Example  Improvement  Processes 

Alternative  improvement  processes  are  supported  by  the  SAIF  architecture,  in  order  to 
provide  adaptability  to  a  variety  of  process  improvement  approaches. 

One  example  of  an  improvement  process  is  the  Plan-Do-Check-Act  (PDCA)  model, 
attributed  to  Shewart.  This  model  is  shown  in  Figure  9  below. 
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Using  this  model,  the  SAIF  would  organize  the  applicable  technologies  and  guidance 
material  into  the  following  categories: 

•  plan  phase 

•  do  phase 

•  check  phase 

•  act  phase 

This  organization  allows  the  user  to  focus  on  the  phase  of  process  improvement  in  which  he 
or  she  is  interested,  and  find  guidance  and  practices  appropriate  to  that  phase  of  the  program. 


Figure  9:  Plan-Do-Check-Act  Mode I  for  Process  Improvement 

SEI’s  IDEAL  model  for  process  improvement,  shown  in  Figure  10,  would  represent  an 
alternative  improvement  process. 


Using  the  IDEAL  model  as  the  chosen  improvement  process  would  yield  the  following 
information  categories,  organized  around  the  IDEAL  model: 

•  initiating  phase 

•  diagnosing  phase 

•  establishing  phase 

•  acting  phase 

•  learning  phase 
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Figure  10:  IDEAL  Model 
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Appendix  C:  Criteria  for  Judging 
“Effective”  Acquisition-Management 
Practices 


Name  of  Practice:  Brief  nomenclature,  if  available.  It  is  important  to  distinguish  between  a 
practice  and  guidance.  For  our  purposes,  we  define  a  practice  as  a  customary  or  established 
way  of  performing  an  activity  or  a  set  of  activities;  an  “effective”  practice  is  a  customary 
way  that  results  in  the  timely  accomplishment  of  goals  supported  by  the  activity  within  a 
desired  cost  range.  Note  that  one  goal  may  be  supported  by  several  activities  or  practices. 
Goals  toward  process  improvement  are  dependent  on  the  domain  and  reference  model 
chosen.  The  distinction  between  a  process  and  a  practice  is  as  follows:  a  process  describes  a 
set  of  activities  performed  for  a  given  purpose  (or  “what”  to  do),  while  a  practice  describes 
“how”  to  do  an  activity  or  a  set  of  activities  within  a  process. 

Description  of  Practice:  Brief  abstract  describing  the  practice.  This  description  is  especially 
needed  if  the  practice  does  not  have  an  accepted  name  or  nomenclature  outside  of  the  user’s 
domain.  This  description  answers  the  questions,  “What  does  the  practice  do?”  and  “How  is 
the  practice  performed?” 

Purpose:  Answers  the  questions  “Why  is  the  practice  needed?”  and  “Who  uses  the 
practice?” 

Process  Areas:  Process  areas  where  the  practice  is  applicable  (where  does  it  fit  in  a  given 
reference  model?);  applicable  SA-CMM  KPAs  and  activities  will  be  the  reference  model  of 
choice.  Later,  an  acquisition-instantiated  version  of  the  IDEAL  model  might  be  used  or 
possibly  the  IEEE  Recommended  Practice  for  Software  Acquisition  process  areas;  possibly 
Software  Program  Managers’  Network  (SPMN)  Best  Practices. 

Functional  Domain:  Functional  domain(s)  where  the  practice  is  applicable  from  the 
following  eight  Government  acquisition  discipline  areas:  acquisition  management,  system 
engineering,  software  acquisition  management,  test  and  evaluation,  manufacturing  and 
production,  acquisition  logistics,  business/cost  estimating  and  financial  management,  and 
contract  management. 

Acquisition  Life-Cycle  Phase:  Life-cycle  phase(s)  where  the  practice  is  applicable  from  the 
following  five  areas:  Pre-Milestone  0;  Concept  Exploration  or  Phase  0;  Program  Definition 
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and  Risk  Reduction  or  Phase  I;  Engineering  and  Manufacturing  Development  or  Phase  II; 
Production,  Fielding/Deployment,  and  Operational  Support  or  Phase  ID. 

Resources:  Extraordinary  resources  required  to  implement  the  practice;  includes  costs, 
people,  skills/training  requirements,  environmental  or  domain  constraints,  functional 
interfaces,  etc. 

Usage/Limitations:  Provides  the  context  for  proper  use  of  the  practice  including  any 
limitations  on  its  use.  If  the  practice  has  security  or  proprietary  limitations,  document  the 
practice  as  much  as  allowed  and  indicate  the  limitation  here.  Note:  we  assume  the  practice  is 
oriented  toward  process  improvement. 

Pros:  Any  advantages  of  applying  the  practice  within  the  scope  of  its  intended  use  (i.e., 
known  positive  effects  from  adopting  the  practice). 

Cons:  Any  disadvantages  of  applying  the  practice  within  the  scope  of  its  intended  use  (i.e., 
known  negative  effects  from  adopting  the  practice). 

Origin  (optional):  Original  source  or  brief  background  of  the  practice,  if  known.  If  the 
practice  is  already  universally  accepted  as  “effective,”  what  criteria  were  used  or  what 
evolution  occurred  to  make  it  “effective”? 

Supporting  Tools,  Techniques,  and  Technologies:  Tools,  techniques,  and  /or  technologies 
that  support  the  practice,  if  any;  the  prerequisite  technologies  that  allow  the  practice  to  be 
implemented  efficiently.  A  supporting  technology  for  one  practice  could  be  a  practice  in  its 
own  right  (e.g.,  Earned  Value  Management  System  [EVMS]  is  a  practice  supported  by 
several  software  tools,  one  of  which  is  wlnsight;  the  wlnsight  tool  is  a  practice  in  itself  for 
cost  and  schedule  project  management). 

Recommendations  for  Change:  Any  known  existing  recommendations  for  future 
modifications,  tailoring,  or  extensions  of  the  practice. 

Reference,  Information  Sources,  Point  of  Contact:  Documentation  and  bibliographic 
information  or  link  to  information  sources.  Also  include  current  point  of  contact  for  the 
practice  if  different  from  information  source. 

Last  Modified:  Date  on  which  this  practice  description  was  last  modified,  if  known. 
Estimate  if  unknown  and  indicate  the  date  is  an  estimate. 

Acceptance/Quality  Criteria:  Description  of  the  criteria  that  are  used  to  judge  the 
successful  application  and  general  acceptance  of  the  practice  by  the  acquisition  community. 
Criteria  below  must  be  objective  and  verifiable: 
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•  Software-intensive  systems  acquisition  domain:  Practice  must  fit  into  the  chosen 
reference  model  defined  above  (i.e.,  SA-CMM). 

•  Legal  and  regulatory  compliance:  Practice  must  comply  with  applicable  laws  and 
regulations  within  the  functional  domain  (e.g.,  federal  acquisition  regulation  [FAR]  for 
DoD,  ISO,  standards,  etc.). 

•  Well  documented:  Unambiguous  understanding  based  on  practice  description  and 
supporting  documentation;  documentation  defines  goals,  steps,  tasks,  activities,  inputs, 
outputs,  measurements,  and  verification  required. 

•  Maturity/successful  application  history:  Describe  how  welldeveloped  and  understood 
the  practice  is;  give  past  and  present  projects  or  efforts  where  the  practice  has  been 
successfully  used  and  points  of  contact  for  each  successful  project.  A  mature  practice 
usually  implies  successful  application  on  several  projects  or  efforts  each  over  a 
significant  timeframe. 

•  Overall  return  on  investment  (ROI):  Practice  incorporates  a  way  of  realizing  a  return 
on  investment  that  can  be  measured  or  equated  to  a  positive  long-term  goal  (e.g.,  dollars 
saved,  increased  productivity,  increased  quality  of  product  or  service,  increased  morale). 
A  practice  may  require  a  short-term  cost  growth  that  results  in  a  long-term  cost  savings, 
so  ROI  should  be  interpreted  with  the  overall  life  cycle  of  the  project  or  effort  in  mind. 

•  External  reviewers/evaluators:  Names  of  experts  who  have  reviewed  the  practice 
description  and  evaluated  the  practice. 

Note:  The  SEI  Technology  Reference  Guide  Quality  Measures  Taxonomy  contains  additional 
criteria  that  may  be  applicable  as  well  [Foreman  97]. 

Example  Using  Continuous  Risk  Management 

Name/Description  of  Practice:  Continuous  Risk  Management 

Purpose:  A  paradigm  for  managing  project  risks 

Process  Area/  Functional  Domain/Acquisition  Life  Cycle:  All 

Resources:  CRM  training  for  implementers 

Usage/Limitations:  Nothing  significant.  Security  environments  may  impede  open 
communications. 

Pros/Cons 

Pros:  Better  understanding  and  mitigation  of  project  risks 

Cons:  Time  investment  required  to  get  up  to  speed 

Origin:  SEI/CMU  Risk  Program;  documented  in  CRM  Guidebook  published  Sept.  1996 

Supporting  Tools,  Techniques,  and  Technologies:  Documented  in  appendices  of  CRM 
Handbook  which  collectively  support  the  paradigm 


CMU/SEI-98-TR-003 


33 


Recommendation  for  Change:  CRM  is  flexible  and  adaptable,  typically  tailored  to  an 
organization’s  processes  and  procedures;  recommended  changes  are  collected  by  the  SEI  for 
the  next  edition  of  the  Guidebook. 

Reference,  Information  Sources,  Point  of  Contact:  See  bibliography  in  the  CRM 
Guidebook. 

Point  of  Contact:  Ray  Williams,  SEI/CMU,  phone  412-268-7614 
Last  Modified:  One  publication  so  far  in  Sept  1996 
Acceptance/Quality  Criteria: 

Applicable  to  Project  Management  and  Acquisition  Risk  Management  KPAs  of  SA-CMM. 
No  regulatory  limitations.  Well  documented  in  guidebook;  training  available.  Successful 
application  on  a  limited  number  of  projects  given  recent  paradigm  introduction.  ROI  is 
dependent  on  specific  application  and  implementation  of  ROI  metrics  within  project.  CRM 
Guidebook  authors  and  project  personnel  act  as  internal  reviewers;  the  guidebook  lists 
experts  outside  the  SEI  who  served  as  external  reviewers  (Charette,  Hall,  etc.). 
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Glossary 


acquisition 

acquisition 

organization 


capability  maturity 
model 


process 


process  capability 


Software  Acquisition 

Improvement 

Framework 


software  acquisition 
process 


The  process  of  obtaining  through  contract. 

That  entity  that  has  the  oversight  responsibility  for  the  software 
acquisition  project  and  that  may  have  purview  over  the 
acquisition  activities  of  a  number  of  projects  or  contract  actions. 

A  description  of  the  stages  through  which  organizations  evolve  as 
they  define,  implement,  measure,  control,  and  improve  their 
processes.  The  model  provides  a  guide  for  selecting  process 
improvement  strategies  by  helping  an  organization  to  determine 
its  current  process  capabilities  and  identify  the  issues  most 
critical  to  quality  and  process  improvement. 

A  set  of  activities  performed  for  a  given  purpose  (e.g.,  the 
software  acquisition  process). 

The  range  of  expected  results  that  can  be  achieved  by  following  a 
process.  (See  process  performance  for  contrast.) 

A  system  consisting  of  a  disciplined  process  for  improvement,  a 
reference  model,  and  supporting  artifacts  that  are  integrated  to  be 
mutually  supportive  and  facilitate  the  improvement  of  an 
acquisition  process. 

A  set  of  activities,  methods,  practices,  and  transformations  that 
people  use  to  acquire  software  and  the  associated  products. 
acquisition  organization  s  standard  software  acquisition  process 
-  The  acquisition  organization’s  fundamental  software  acquisition 
process  that  guides  the  establishment  of  each  project’s  defined 
software  acquisition  process,  project’s  defined  software 
acquisition  process  -  The  project’s  tailored  version  of  the 
acquisition  organization’s  standard  software  acquisition  process. 
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software  acquisition 
project 


software  acquisition- 
related  group 


software 

architecture 

software-related 

contractual 

requirements 

software  support 


solicitation  package 


traceability 


An  undertaking  that  is  focused  on  acquiring  the  software 
components  and  associated  documentation  of  a  system.  A 
software  project  may  be  part  of  a  project  building  a 
hardware/software  system. 

A  collection  of  individuals  (both  managers  and  technical  staff) 
representing  a  software  discipline  that  supports,  but  is  not 
directly  responsible  for,  software  acquisition.  Examples  of 
software  disciplines  include  software  configuration  management 
and  software  quality  assurance. 

The  organizational  structure  of  the  software  or  module  [IEEE- 
STD-610]. 

All  technical  and  nontechnical  requirements  related  to  the 
software  portion  of  the  acquisition. 

The  process  of  modifying  a  software  system  or  component  after 
delivery  to  correct  faults,  improve  performance  or  other 
attributes,  or  adapt  to  a  changed  environment  [IEEE-STD-610]. 

Information  distributed  when  seeking  suppliers  for  a  particular 
acquisition.  This  package  describes  to  interested  bidders  what  the 
requirements  are,  how  to  prepare  their  proposals,  how  proposals 
will  be  evaluated,  and  when  to  submit  their  proposals.  It  is 
sometimes  called  a  request  for  proposal  (RFP). 

The  ability  to  trace,  in  both  the  forward  and  backward  directions, 
the  lineage  of  a  requirement  from  its  first-level  inception  and 
subsequent  refinement  to  its  implementation  in  a  software 
product  and  the  documentation  associated  with  the  software 
product. 
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